(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date 
26 April 2001 (26.04.2001) 




PCT 



(10) International Publication Number 

WO 01/29817 Al 



(51) International Patent Classification 7 : G09G 5/20 

(21) International Application Number: PCT/US00/29037 

(22) International Filing Date: 19 October 2000 (19.10.2000) 

(25) Filing Language: English 

(26) Publication Language: English 
(30) Priority Data: 



60/160,397 



19 October 1999 (19.10.1999) US 



(71) Applicant (for all designated States except US): INTEN- 
SYS CORPORATION [US/US]; 2033 Gateway Place, 
Suite 600, San Jose, CA 951 10 (US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): GHULOUM, An- 
war, M. [US/US]; 547 McCarty Avenue, Mountain View, 
CA 94041 (US). TAM, Pak, Y. [US/US]; 4355 Renais- 
sance Drive, #102, San Jose, Ca 95134 (US). OUYANG, 
Xiaolong [US/US]; 969 Lawrence Lance, Palo Alto, CA 
94303 (US). LAM, Edmund, Y [GB/US]; 425 Grant Av- 
enue, #24, Palo Alto, CA 94306 (US). 



(74) Agent: HODES, Alan, S.; McCutchen, Doyle, Brown & 
Enersen, LLP, Three Embarcadero Center, San Francisco, 
CA 941 11 (US). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU, CZ, 
DE, DK. DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, 
HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, LK, LR, 
LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, 
NO, NZ, PL, PT, RO, RU, SD, SE, SG. SI, SK, SL, TJ, TM, 
TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL. SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU. TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, 
IT, LU, MC, NL, PT, SE), OAPI patent (BF, BJ, CF, CG, 
CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— With international search report. 

— Before the expiration of the time limit for amending the 
claims and to be republished in the event of receipt of 
amendments. 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



=E (54) Title: IMPROVING IMAGE DISPLAY QUALITY BY ADAPTIVE SUBPIXEL RENDERING 



00 

o 
O 



Receiving pixed data 
representing an image 
4Q1 



Determining an image feature 
402 



I 



Generating a subpixed 
drive correction signal 
403 



i 



Generating a drive signal 

m 



(57) Abstract: A method for displaying an image comprises receiving pixel data 
representing an image, determining an image feature in the received pixel data, 
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IMPROVING IMAGE DISPLAY QUALITY BY ADAPTIVE SUBPIXEL 

RENDERING 

5 PRIORITY 

This application claims priority to Provisional Application No. 60/160,397, 
filed on October 19, 1999 , which is incorporated herein. 
. FIELD OF THE INVENTION 

The present invention relates to displaying images. In particular, the present 
10 invention relates to improving image-display quality by adaptive subpixel rendering. 
BACKGROUND 

Multicolor displays typically use pixels that are constructed from discrete red, 
green and blue subpixels. One example of such multicolor displays, liquid crystal 
displays (LCDs) have three spatially-offset color components including red, green and 
15 blue subpixels. One example of color components in an LCD screen is shown in 
Figure 1, which consists of rectangular areas formed by arranged the red, green and 
blue components in a horizontal line. As can be seen from Figure 1, pixel 101 in this 
example consists of red subpixel 101a, green subpixel IOlb and blue subpixel 101c. 
Of course, other arrangements are possible, such as those described by S. Tsuruta, et 
20 al M in "Color Pixel Arrangement Evaluation for LC-TV" ( 1 985 International Display 
Research Conference, pp. 24-26). 

Each of the color components can be individually controlled by an LCD 
controller to produce different shades of color. Because each set of the color 
components belongs to a single pixel, however, in many image processing algorithms, 
25 including image rendering or rasterization, the subpixels are treated as if they co- 
locate in a single coordinate with many similar attributes. This can lead to jagged 
lines and inferior image display quality. Figure 2 shows an example of a diagonal line 
displayed using a typical pixel-rendering scheme. As can be seen from this picture, 
the line is jagged. Ways exist to address this problem, including methods of anti- 
30 aliasing. A typical anti-aliased diagonal line is shown in Figure 3. Using anti- 
aliasing, pixels adjacent to the blackened pixel are displayed in various shades of 
gray, thereby allowing the eye to smooth the line. Unfortunately, anti-aliasing 
diminishes image sharpness, and can result in an unclear, or blurry image. 



Other methods exist for improving image displays, but each of these methods 
have undesirable drawbacks. One method, for example, is shown in U.S. Patent No. 
5,341,153, entitled "Method and Apparatus for Displaying a Multicolor Image." The 
invention in that patent attempts to deal with the problem of single-colored subpixels. 
That invention, however, attempts to correct for subpixel displacement, rather than 
using the displacement as a characteristic to improve a displayed image. 

Another method is found in U.S. Patent No. 6,028,646, titled "Color Image 
Enhancement Device for Video Display Appliance." The invention in that patent 
attempts to provide a color-image enhancement device for improving the sharpness of 
the color image by adjusting the image's saturation. For the purposes of that 
invention, however, the subpixels are assumed to co-locate, and color space 
transformation is performed to convert the image's representation from red, green and 
blue to luminance, hue and saturation. As such, the advantages of using display 
attributes are typically not explored. 

Thus, because of the drawbacks in typical pixel displays, a need is shown for 
improved image rendering that can use subpixels to increase image clarity and 
resolution. 

SUMMARY OF THE INVENTION. 

To alleviate the problems in known systems, displaying images using adaptive 
subpixel rendering is discussed herein. 

Embodiments of the present invention allow for improved display quality by 
employing methods and devices that perform adaptive subpixel rendering. In one 
embodiment of the present invention, a method for displaying an image comprises 
receiving pixel data representing an image, determining an image feature in the 
received pixel data, and generating a subpixel drive correction signal based on the 
determined image. In another embodiment of the present invention, the subpixel 
drive correction signal is based on an attribute of a display device. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a representation of a known subpixel structure. 

Figure 2 is a representation of an example of a diagonal line displayed using a 
typical pixel-rendering scheme. 

Figure 3 is a representation of an example of a diagonal line displayed using 
an anti-aliasing scheme. 

Figure 4 is a flow chart of an embodiment of the present invention. 



Figure 5 is a flow chart of calculations, according to an embodiment of the 
present invention. 

Figure 6 is a block diagram of an apparatus according to an embodiment of the 
present invention, including a processor, a memory and a port. 

Figure 7 is a block diagram of another apparatus embodiment of the present 
invention, including an I/O Buffer, and Address Counter, a RAM, a Timing 
Generator, a MPU Interface, a Command Decoder, a Subpixel Feature Enhancer, and 
an Apparatus Driver. 

Figure 8 is a representation of a black vertical line, illustrating an embodiment 
of the present invention. 

Figure 9 is a representation of a diagonal line, according to an embodiment of 
the present invention. 
DETAILED DESCRIPTION 

Embodiments of the present invention provide improved image-display quality 
using adaptive subpixel rendering. If desired, embodiments of the present invention 
can use attributes of a display device to provide the improved image-display quality. 

As a contextual example, but not the only example, consider a multicolor 
display constructed from discrete red, green and blue components, such as a Liquid 
Crystal Display (LCD). 

Figure 4 is a flow chart of an embodiment of the present invention. In this 
embodiment, pixel data representing an image is received at step 401 . At step 402, an 
image feature in the received pixel data is determined. Using the determined image 
feature, at step 403, a subpixel drive correction signal is generated. Once the subpixel 
drive correction signal is generated, at step 404, a drive signal based on the subpixel 
drive correction signal is generated. 

In one embodiment of the present invention, the subpixel drive correction 
signal can be based on an attribute or attributes of a given display device. For 
example, but not the only example, a display device can be an LCD. The attributes 
can include, but are not limited to, subpixel configurations such as the relative 
position of the subpixels, the size of the subpixels, the shape of the subpixels, the 
geometry of the subpixels and the orientation of the subpixels. For example, referring 
back to Figure 1 , as can be seen, the red, green and blue subpixels have a spatial offset 
from one another. These subpixel spatial offsets can be used to increase the spatial 



resolution of the image by a factor of N, where N is the number of color components 
for a given pixel. 

The finite size and shape of the subpixels are also attributes that can be used to 
enhance the image because they determine the size and shape of the image details that 
can best be represented by the subpixels. 

The geometry and the orientation are both concerned with the arrangement of 
the subpixels. Different arrangements allow our eyes to perceive sharper details at 
certain angles and frequencies, when the image is rendered at a subpixel level. 

In another embodiment of the present invention, the attribute of "intensity 
mapping" is used to enhance the image. Because the color components can be 
controlled individually by a pixel drive signal, by treating them as separate subpixels 
it is possible to use dynamic intensity mapping to further improve image quality. 

In one embodiment of the present invention, the pixel data received at step 401 
can include subpixel data, and can be rendered or rasterized data. For the purposes of 
the present invention, the term "rasterization" means determining pixel values from 
input geometric primitives. 

The image feature detection performed at step 402 can be performed in any 
known way. In one embodiment of the present invention, to make use of the subpixel 
attributes, after the information of the rendered image is passed on to, for example, 
but not the only example, a display controller, it is passed through a few stages of 
calculation. Figure 5 is a flow chart of such calculations, according to an embodiment 
of the present invention. At step 501 , an NxM square, or window, of pixels is 
identified or input, where N and M can include, for example (but not the only 
example) from 3 to 9 pixels. One skilled in the art will realize, of course, that the 
values of N and M can be arbitrarily small or arbitrarily large, depending on the image 
feature detection to be performed, or depending on the end result that is desired. The 
range "3 to 9 pixels" is only an example to illustrate what is meant by "an NxM 
square." 

At step 502, an edge in the input image block is detected, and at step 503, an 
edge metric is calculated The edge metric can include, but is not limited to, 
information about the edge location, the edge orientation, and contrast The detection 
and calculation can be performed by any appropriate operation. For example, but not 
the only example, edge detection and calculation can be performed by using the Sobel 
or Laplacian filter for edge detection. Other examples are described in Gonzalez and 
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Woods, Digital Image Processing (Addison-Wesley, 1992), the relevant portions of 
which are hereby incorporated. The calculations can be performed on, for example, 
but not the only example, a single instruction, multiple data (SIMD) array processor. 
At step 504, pixel and subpixel attributes are combined with the edge metric. 
5 At step 505, the image features can be rendered or rasterized appropriately with 
subpixel accuracy for any particular color attribute or component geometry in a 
display device. This rendering or rasterization can be performed in software using 
any appropriate algorithm. 

One skilled in the art will recognize that the order of the flow chart above, and 

10 all the flow charts in this document (including all claims), is not meant to imply a 
necessary order. For the purposes of the present invention, the steps of the flow 
charts and claims in this document can be performed in any order practicable. 

Figure 6 is a block diagram of an apparatus according to an embodiment of the 
present invention. In one embodiment of the present invention, processor 601 is 

is coupled to memory 602 and port 603, and memory 602 stores instructions adapted to 
be executed by processor to perform a method embodiment of the present invention. 
For example, but not the only example, memory 602 stores instructions adapted to be 
executed by processor 601 to receive a rasterized image, detect a feature in the 
rasterized image, and then enhance the detected feature on a subpixel level. 

20 For the purposes of this application, memory includes any computer-readable 

medium capable of storing instructions adapted to be executed by a processor. Some 
examples of such media include, but are not limited to, RAM, ROM, floppy disks, 
CDROM, magnetic tape, hard drives, and any other device that can store digital 
information. In one embodiment, the instructions are stored on the medium in a 

25 compressed and/or encrypted format. As used herein, the phrase "adapted to be 
executed by a processor" is meant to encompass instructions stored in a compressed 
and/or encrypted format, as well as instructions that have to be compiled or installed 
by an installer before being executed by the processor. 

Figure 7 is a block diagram of another apparatus embodiment of the present 

30 invention. In this block diagram, I/O Buffer 701 is coupled to Address Counter 702. 
Address Counter 702 is coupled to RAM 703, which in turn is coupled to Timing 
Generator 704, Command Decoder 705, and Subpixel Feature Enhancer 706. 
Subpixel Feature Enhancer 706 and Timing Generator 704 are both coupled to 
Apparatus Driver 707, and Command Decoder 705 is coupled to MPU Interface 708. 



WO 01/29817 



PCT/US00/29O37 



I/O Buffer 701 can receive rendered or rasterized data representing an image. 
Address Counter 702 can direct which of the data in I/O Buffer 701 is to be 
transferred to RAM 703 in the controller. Timing Generator 704 can provide a timing 
pattern for the entire controller, or some portion thereof. MPU Interface 708 can 
5 direct external commands to Command Decoder 705, wherein the commands can be 
subsequently directed to data stored in RAM 703. 

In Subpixel Feature Enhancer 706, data representing an image block can be 
fed from RAM 703. Subpixel Feature Enhancer 706 can then determine particular 
image features in the received pixel data, such as the existence of an edge. If a 
10 particular feature is identified, the characteristics of the feature are also computed in 
Subpixel Feature Enhancer 706, such as the orientation and contrast of the edge. 
Taking into account the attributes of the display controller, Subpixel Feature Enhancer 
706 can generate a subpixel drive correction signal to enhance the detected features. 
The resulting enhanced image data are fed to the display driver which interacts with 
15 the actual image display. 

The following examples are comparisons of various image renderings 
according to embodiments of the present invention, as compared with image 
renderings without applying embodiments of the present invention. 

Consider Figure 8, which displays the simple case of a black vertical line, with 
20 a width of exactly one pixel wide. As can be seen in 8(a), the vertical line is displaced 
by one-third with respect to the pixel grid. When the line is rendered on a pixel level, 
considering the situation where white corresponds to maximum intensity and black 
corresponds to minimum intensity, the intensity of the left column is reduced to two- 
third while the intensity of the second column is reduced to one-third, to give the 
25 impression to the eye that together they form a black vertical line, as shown in 8(b). 

If such an image is displayed on an LCD with an arrangement of the subpixels similar 
to the subpixels found in Figure 1, the individual subpixel intensities are adjusted as 
in 8(c), where the red, green, and blue subpixels corresponding to the pixels in the left 
column are all reduced by two-thirds, and those corresponding to the pixels in the 
30 right column are all reduced by one-third. One skilled in the art will appreciate that, 
for the sake of clarity in the diagram, the red, green, and blue subpixels that 
correspond to the white pixels in the background are not shown. The image 
sharpness, however, can be significantly enhanced with subpixel level rendering, 
taking into account the subpixel attributes. The result is illustrated in 8(d), where the 
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blue subpixels corresponding to the pixels in the left column and the red and green 
subpixels corresponding to the pixels in the right column are turned to minimum 
intensity. Because eyes would average a set of red, green, and blue pixels together, 
such a scheme would produce a black vertical line without color artifacts. 
5 Such a subpixel rendering scheme is also helpful for edges tilted at an acute or 

obtuse angle with respect to the horizontal or vertical grid. Conventionally, as 
discussed in the BACKGROUND, above, such a line is rendered using a standard 
anti-aliasing operation performed to preserve object edges. Conventional anti-aliasing 
schemes typically gray out the whole of some neighboring pixels to enhance the 

10 quality of the rendered fine line. This is schematically illustrated in Figures 2 and 3, 
as discussed above, and described in details in J. Foley et al. Computer Graphics: 
Principles and Practice, Second Edition in C (Addison- Wesley, 1 995). In figure 2, 
one can see a tilted line rendered without anti-aliasing, while Figure 3 shows the same 
line rendered with traditional anti-aliasing. 

15 By taking into account the display attributes, however, the image quality can 

be improved in a way similar to the way the vertical line was improved in Figure 8. 
This is illustrated in figure 9. At 9(a), one can see the conventional anti-aliasing 
scheme together with the color components. When the algorithm calls for a graying 
of the neighborhood pixels, all 3 color components are reduced in intensity by the 

20 same amount At 9(b), on the other hand, one can see the subpixel anti-aliasing 

scheme. At neighborhood locations of the fine line that requires diminished intensity, 
one can selectively reduce the intensities of the red, green, or blue components to 
. minimize the jaggedness and color distortions of the rendered line. This results in a 
smoother and clearer line than would be available from the conventional method. 

25 As one skilled in the art will appreciate, subpixel-level feature enhancement 

schemes for image display are not limited to anti-aliasing applications. For example, 
but not the only example, another particular advantageous application is for low-end 
displays, such as palm-top organizers and mobile phones, in which maximum 
resolution texts should be displayed with as small a number of pixels as possible. For 

30 such devices, screen resolution is usually very limited. Thus, if the image rendering 
from the host computer is performed on a pixel level, many fine lines may be lost or 
difficult to represent By using subpixel rendering, on the other hand, and taking into 
account the display attributes, resolution can be increased by up to N times, where N 
is the number of color components for a given pixel. Using similar techniques as 
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described above, fine lines can be represented more precisely with subpixel level 
rendering. 

The present invention has been described in terms of several embodiments 
solely for the purpose of illustration. Persons skilled in the art will recognize from 
5 this description that the invention is not limited to the embodiments described, but 
may be practiced with modifications and alterations limited only by the spirit and 
scope of the appended claims. 
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1 WHAT IS CLAIMED IS 

2 1. A method for displaying an image, the method comprising: 

3 (a) receiving pixel data representing an image; 

4 (b) determining an image feature in the received pixel data; and 

5 (c) generating a subpixel drive correction signal based on the determined 

6 image. 

1 2 . The method of claim 1 , wherein said generating a subpixel drive correction 

2 signal is further based on an attribute of a display device. 

1 3. The method of claim 2, wherein the attribute is a subpixel configuration. 

1 4. The method of claim 2, wherein the attribute is a relative position of sub-pixels 

2 within a pixel. 

1 5. The method of claim 2, wherein the attribute is a geometry of subpixels within 

2 a pixel. 

1 6. The method of claim 2, wherein the attribute is an orientation of subpixels 

2 within a pixel. 

1 7. The method of claim 2, wherein the attribute is a size of subpixels within a 

2 pixel. 

1 8 . The method of claim 1 , further comprising: 

2 (a) generating a drive signal based on the subpixel drive correction signal. 

1 9. The method of claim 2, further comprising: 

2 (a) generating a drive signal based on the subpixel drive correction signal. 

1 10. A computer readable medium storing instructions adapted to be executed on a 

2 processor, the instructions comprising: 

3 (a) receiving pixel data representing an image; 

4 (b) detennining an image feature in the received pixel data; and 

5 (c) generating a subpixel drive correction signal based on the determined 

6 image. 
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1 11. The computer readable medium of claim 1 0, wherein the instruction for 

2 generating a subpixel drive correction signal is further based on an attribute of 

3 a display device. 

1 12. The computer readable medium of claim 1 1 , wherein the attribute is a subpixel 

2 configuration. 

1 13. The computer readable medium of claim 1 1 , wherein the attribute is a relative 

2 position of subpixels within a pixel 

1 14. The computer readable medium of claim 1 1 , wherein the attribute is a 

2 geometry of subpixels within a pixel. 

1 1 5. The computer readable medium of claim 1 1 , wherein the attribute is an 

2 orientation of subpixels within a pixel. 

1 16. The computer readable medium of claim 11, wherein the attribute is a size of 

2 subpixels within a pixel. 

1 17. The computer readable medium of claim 1 0, further comprising: 

2 (a) generating a drive signal based on the subpixel drive correction signal. 

1 18. The computer readable medium of claim 1 1 , further comprising: 

2 (a) generating a drive signal based on the subpixel drive correction signal. 

1 19. An apparatus for displaying an image, the apparatus comprising: 

2 (a) means for receiving pixel data representing an image; 

3 (b) means for determining an image feature in the received pixel data; and 

4 (c) means for generating a subpixel drive correction signal based on the 

5 determined image. 

1 20. The apparatus of claim 19, wherein said means for generating a subpixel drive 

2 correction signal is further based on an attribute of a display device. 

1 21. The apparatus of claim 1 9, further comprising: 

2 (a) means for generating a drive signal based on the subpixel drive 

3 correction signal. 
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1 22. The apparatus of claim 20, further comprising: 

2 (a) means for generating a drive signal based on the subpixel drive 

3 correction signal. 

1 23. A method for displaying an image, the method comprising: 

2 (a) receiving a rasterized image; 

3 (b) detecting a feature in the rasterized image; and 

4 (c) enhancing the detected feature on a subpixel level. 

1 24. The method of claim 23, wherein said enhancing is based on an attribute of a 

2 display device. 

1 25. A computer readable medium storing instructions adapted to be executed on a 

2 processor, the instructions comprising: 

3 (a) receiving a rasterized image; 

4 (b) detecting a feature in the rasterized image; and 

5 (c) enhancing the detected feature on a subpixel level. 

1 26. The computer readable medium of claim 25, wherein said enhancing is based 

2 on an attribute of a display device. 

1 27. An apparatus for displaying images, the apparatus comprising: 

2 (a) means for receiving a rasterized image; 

3 (b) means for detecting a feature in the rasterized image; and 

4 (c) means for enhancing the detected feature on a subpixel level. 

1 28. The apparatus of claim 27, wherein said means for enhancing is based on an 

2 attribute of a display device. 
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FIG. 1 (Prior Art) 
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(a) Without anti-aliasing 



FIG. 2 




(b) With anti-aliasing 



FIG. 3 
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(a) Ideal line 



FIG. 8A 




(c) Pixel level rendering in LCD 



FIG. 8C 




(b) Pixel level rendering 



FIG. 8B 




(d) Subpixel level rendering 
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